Human-like agents have long been one of the goals in pursuing artificial intelligence. Although reinforcement learning (RL) has achieved superhuman performance in many domains, relatively little attention has been focused on designing human-like RL agents. As a result, many reward-driven RL agents often exhibit unnatural behaviors compared to humans, raising concerns for both interpretability and trustworthiness. To achieve human-like behavior in RL, this paper first formulates human-likeness as trajectory optimization, where the objective is to find an action sequence that closely aligns with human behavior while also maximizing rewards, and adapts the classic receding-horizon control to human-like learning as a tractable and efficient implementation. To achieve this, we introduce Macro Action Quantization (MAQ), a human-like RL framework that distills human demonstrations into macro actions via Vector-Quantized VAE. Experiments on D4RL Adroit benchmarks show that MAQ significantly improves human-likeness, increasing trajectory similarity scores, and achieving the highest human-likeness rankings among all RL agents in the human evaluation study. Our results also demonstrate that MAQ can be easily integrated into various off-the-shelf RL algorithms, opening a promising direction for learning human-like RL agents.

Human-like agents have long been one of the goals in pursuing artificial intelligence. Although reinforcement learning (RL) has achieved superhuman performance in many domains, relatively little attention has been focused on designing human-like RL agents. As a result, many reward-driven RL agents often exhibit unnatural behaviors compared to humans, raising concerns for both interpretability and trustworthiness. To achieve human-like behavior in RL, this paper first formulates human-likeness as trajectory optimization, where the objective is to find an action sequence that closely aligns with human behavior while also maximizing rewards, and adapts the classic receding-horizon control to human-like learning as a tractable and efficient implementation. To achieve this, we introduce Macro Action Quantization (MAQ), a human-like RL framework that distills human demonstrations into macro actions via Vector-Quantized VAE. Experiments on D4RL Adroit benchmarks show that MAQ significantly improves human-likeness, increasing trajectory similarity scores, and achieving the highest human-likeness rankings among all RL agents in the human evaluation study. Our results also demonstrate that MAQ can be easily integrated into various off-the-shelf RL algorithms, opening a promising direction for learning human-like RL agents. Our code is available at https://rlg.iis.sinica.edu.tw/papers/MAQ.

Driverless cars are beginning to display human-like behaviors like impatience on the roads, in a sign of increased intelligence in the robotaxis. The chilling development was identified by University of San Francisco engineering Professor William Riggs, who has been studying Waymo cars since their inception. On a journey with a reporter from the San Francisco Chronicle, the pair noticed the Waymo they were traveling in crept to a rolling start at a pedestrian crossing before the person had reached the other footpath. The subtle movement was reminiscent of the way humans act behind the wheel, but a strange occurrence for the robotic Waymo, which prides itself on being safer than a driver because it errs on the side of caution and leaves no room for human error. The action of letting the foot gently off the break moments before they should to allow the car to begin creeping forward at a rolling pace displays a sense of impatience - a human reaction not previously seen in the robotic cars.

Scientists have observed another human-like behavior among elephants - they call each other by name. Researchers from Colorado State University (CSU) recorded 470 unique noises from elephants in Kenya, capturing different rumbles and pitches. Using machine learning, the team found the calls contained a unique tune depending on which elephant they were communicating with. To test their theory that these noises corresponded with different names, the team played them to the herds - and the elephant being named responded by returning a noise or approaching the speaker. The findings suggest elephants may be capable of abstract thinking, making them much more socially complex mammals than previously thought.

Reports of human-like behaviors in foundation models are growing, with psychological theories providing enduring tools to investigate these behaviors. However, current research tends to directly apply these human-oriented tools without verifying the faithfulness of their outcomes. In this paper, we introduce a framework, RealBehavior, which is designed to characterize the humanoid behaviors of models faithfully. Beyond simply measuring behaviors, our framework assesses the faithfulness of results based on reproducibility, internal and external consistency, and generalizability. Our findings suggest that a simple application of psychological tools cannot faithfully characterize all human-like behaviors. Moreover, we discuss the impacts of aligning models with human and social values, arguing for the necessity of diversifying alignment objectives to prevent the creation of models with restricted characteristics.

Generating human-like behavior on robots is a great challenge especially in dexterous manipulation tasks with robotic hands. Scripting policies from scratch is intractable due to the high-dimensional control space, and training policies with reinforcement learning (RL) and manual reward engineering can also be hard and lead to unnatural motions. Leveraging the recent progress on RL from Human Feedback, we propose a framework that learns a universal human prior using direct human preference feedback over videos, for efficiently tuning the RL policies on 20 dual-hand robot manipulation tasks in simulation, without a single human demonstration. A task-agnostic reward model is trained through iteratively generating diverse polices and collecting human preference over the trajectories; it is then applied for regularizing the behavior of polices in the fine-tuning stage. Our method empirically demonstrates more human-like behaviors on robot hands in diverse tasks including even unseen tasks, indicating its generalization capability.

We aim to understand how people assess human likeness in navigation produced by people and artificially intelligent (AI) agents in a video game. To this end, we propose a novel AI agent with the goal of generating more human-like behavior. We collect hundreds of crowd-sourced assessments comparing the human-likeness of navigation behavior generated by our agent and baseline AI agents with human-generated behavior. Our proposed agent passes a Turing Test, while the baseline agents do not. By passing a Turing Test, we mean that human judges could not quantitatively distinguish between videos of a person and an AI agent navigating. To understand what people believe constitutes human-like navigation, we extensively analyze the justifications of these assessments. This work provides insights into the characteristics that people consider human-like in the context of goal-directed video game navigation, which is a key step for further improving human interactions with AI agents.

Originally published on Towards AI the World's Leading AI and Technology News and Media Company. If you are building an AI-related product or service, we invite you to consider becoming an AI sponsor. At Towards AI, we help scale AI and technology startups. Let us help you unleash your technology to the masses. Data and judgment complement AI as core elements of decision-making in war and national security in general. The human-like behavior of AI-based technology raises questions regarding the interfaces between science, technology, and society.

If you ran into a robot in the street, there are a number of crucial giveaways that would help you pick it out from a human crowd -- silicon skin or unblinking eyes, to name a few. Yet, when people cannot see the robots they interact with, this distinction between what is and isn't human becomes much hazier. That's the premise of the Turing test, which helps determine whether or not a robot or A.I. can pass as human. In a new paper, a team of researchers has implemented a variation of this test that looks at behavior variability found in humans and animals to see whether or not mimicking this trait can make robots seem more alive. By introducing variability in reaction time to robots' otherwise rigidly programmed behavior, the team of researchers found that humans can be fooled into believing a robot is flesh and blood.

Recently developed artificial intelligence (AI) models are capable of many impressive feats, including recognizing images and producing human-like language. But just because AI can perform human-like behaviors doesn't mean it can think or understand like humans. As a researcher studying how humans understand and reason about the world, I think it's important to emphasize the way AI systems "think" and learn is fundamentally different to how humans do--and we have a long way to go before AI can truly think like us. Developments in AI have produced systems that can perform very human-like behaviors. The language model GPT-3 can produce text that's often indistinguishable from human speech.